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Abstract 

The paper presents sufficient conditions of predictability for continuous time pro- 
cesses in deterministic setting. We found that processes with exponential decay on 
energy for higher frequencies are predictable in some weak sense on some finite time 
horizon defined by the rate of decay. Moreover, this predictability can be achieved uni- 
formly over classes of processes. Some explicit formulas for predictors are suggested. 
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1 Introduction 

We study pathwise predictability of continuous time processes in deterministic setting and 
in the framework of the frequency analysis. It is well known that certain restrictions on 
frequency distribution can ensure additional opportunities for prediction and interpolation 
of the processes. The classical result is Nyquist-Shannon-Kotelnikov interpolation theorem 
for the band-limited processes. There are related predictability results; see, e.g., Wainstein 
and Zubakov (1962), Beutler (1966), Brown(1969), Slepian (1978), Knab (1981), Papoulis 
(1985), Marvasti (1986), Vaidyanathan (1987), Lyman et al (2000, 2001). These works 
considered predictability of single processes, and the crucial assumption was that the 
processes are band-limited; the predictors were non-robust with respect to small noise in 
high frequencies; see, e.g., the discussion in Chapter 17 from Higgins (1996). 

We study some special weak predictability of continuous time processes. Instead of 
predictability of the original processes, we study predictability of sets of anticausal con- 
volution integrals for a wide enough classes of kernels. This version of predictability was 
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introduced in Dokuchaev (2008) for band-limited processes; it allowed to establish uni- 
form predictability in this weakened sense over classes of band-limited and high-frequency 
processes. In the present paper, we established some predicability for continuous time 
processes with exponential decay of energy on the higher frequencies. It allows to consider 
processes that are not band-limited. More precisely, we obtain a sufficient condition of 
uniform weak predictability on prediction horizon T over some classes of processes with 
exponential decay of energy on higher frequencies to — > ±00, when the energy is decreas- 
ing faster than e~ T l w l An alternative formulation of this condition in time domain is also 
given, The predictors are obtained explicitly in the frequency domain via their transfer 
function. These predictors are defined entirely by the kernel of the convolution integral 
and their choice is independent from the characteristics of the particular input processes. 

2 Problem setting and definitions 

Let x(t) be a currently observable continuous time process, t G R. The goal is to estimate, 
at a current time t, the values y(t) = k(t — s)x(s)ds, where fe(-) is a given kernel, and 
T > is a given prediction horizon. At any time t, the predictors use historical values of 
the observable process x(s)\ s <t. 

We consider only linear predictors in the form y(t) = k(t — s)x(s)ds, where k(-) 
is a kernel that has to be found. We will call k a predictor or predicting kernel. 

Let us describe admissible classes of k and k. 

Let R+ = [0,+oo), C+ = {zeC: Rez > 0}, i = 

For x G L2CR) U Li(R), we denote by X = Tx the function defined on iH as the 
Fourier transform of x; 



If x € Z/2(R), t nen A is defined as an element of L2(R) (more precisely, A(i-) 6 L2(R)). 
For x(-) G 2v2(R) such that x(t) = for t < 0, we denote by Cx the Laplace transform 



Let H r be the Hardy space of holomorphic on C + functions h(p) with finite norm 
\\h\\ H r = sup s>0 \\h(s + zu;)|| MR ), r G [1, +00] (see, e.g., Duren (1970)). 

Definition 1 For T > 0, we denote by JC(T) the set of functions k : R — > R such that 
k(t) =0fort$ [-T, 0] and such that k G Loo(R)- 





(2.1) 
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Definition 2 Let K, be the class of functions k : R — > R suc/t £/taf fc(t) = /or £ < and 
such that K(-) = Ck G H 2 n 

We consider below k G /C(T) and k & )C. 

Definition 3 Ze£ 6e a class of processes ar(-) /rom ^(R) U Li(R). Let r G [1, +oo]. 

(%) W^e say that the class X is L r -predictable in the weak sense with the predic- 
tion horizon T if, for any k(-) G /C(T), i/iere exists a sequence {k m (-)}^^ 1 = 
{k m (-, X, k)}^^ 1 C K, such that 

\\y ~ ym\\L r (n) as m -> +oo Vx £ <Y, 

a f t+T a /"* ~ 

= / - s)x(s)ds, y m (i) = / k m (t - s)x(s)ds. 

J t J — oo 

The process y m (t) is the prediction of the process y(t) which describes depends on 
the future values of process x(s)\ s£ [ t j + T] ■ 

(ii) Let the set T{X) = {X = Fx, x G X} be provided with a norm \\ ■ \\. We say 
that the class X is L r -predictable in the weak sense with the prediction horizon T 
uniformly with respect to the norm \\ • \\, if, for any k(-) G K.(T), there exists a 
sequence {k m (-)} = {k m (-,X,k,\\ ■ ||,e)} C K, such that 

\\y ~ y IU r (R) — ¥ uniformly in {x G X : \\X\\ < 1}. 
Here y(-) and y m (-) are the same as above. 



3 The main result 

For q G {1,2}, let X(q) = X(q,T) be the set of processes x(-) G L 2 (R) ULi(R) such that 

»+oo 



/+oo 
e qT ^\X(iuj)\ q dLU < +oo, X{iuj) = Tx 
-oo 



For Q > 0, set D(Q) = R\(-fi,fi). 
Clearly, if x{-) G X(q,T), then 



/ e qT ^\X(iuj)\ q duj ->■ as n ->■ +oo. 
JD(n) 



It can be seen also that, for any T > 0, the class X(q, T) includes all band-limited processes 
x such that X(ioj) = Tx G L q (R), q G {1, 2}. 



3 



Theorem 1 Let q £ {1,2}. Set r = r(q) = q(q — 1) 1 (in particular, r = oo for q = 1 
and r = 2 /or q = 2). 

(i) The class X(q,T) is L r -predictable in the weak sense with the prediction horizon T. 

(ii) LetU{q) =U(q,T) be a class of processes x(-) G X(q,T) such that 



Then this class U(q, T) is L r -predictable in the weak sense with the prediction horizon 
T uniformly with respect to the norm \\ ■ \\L q (R)- 

Some alternative descriptions and examples of sets U(q,T) are given below. 

The question arises how to find the predicting kernels. In the proof of Theorem [TJ 
a possible choice of the kernels is given explicitly in the frequency domain, i.e., via the 
transfer functions. 

Remark 1 In Dokuchaev (2008), similar weak predictability with infinite horizon was 
introduced and established for models where an ideal low-pass filter exists; the predictors 
used in this paper were different from the ones presented below. Theorem^ allows to extend 
this weak predictability on the case when the filters are not ideal but allow exponentially 
decay of energy on higher frequencies. 

Remark 2 The case when processes k(-) £ L2(R)\L 00 (R) can be also covered. In this 
case, we have to require that x G L2(R). 

On possibility of extrapolation 

The weak predictability introduced in Theorem [TJ does not ensure extrapolation of the pro- 
cesses from U(q,T) in the classical sense; even approximate extrapolation is not guaran- 
teed. Let us explain why. Let q = 2 and T > be given. Assume that the values of the pro- 
cess x(t) 6 U{2, T) are known for t < r. Consider a sequence of kernels {K m }^^ 1 C /C(T) 
that forms a orthonormal basis in the Hilbert space L2(—T, 0) (for instance, one may take 
Fourier series). For any r, the function x(t)|j r t +t] £ ^(r^r + T) can be represented 
as a series x(t) = Y^m=i fmK m {r + T -t), where f m = J^ +T K m (r + T - s)x(s)ds are 
the corresponding Fourier coefficients (the series converges in L2(t,t + T)). Theorem Q] 
ensures that, for any e > 0, there exists a predictor such that the values of f m can be 
predicted at time t with the error less or equal than e for all m. Unfortunately, it does 
not help to predict the summa of infinite series, even if this e is small. 




X(iuj)\ q duj — > as 0, — > +oo uniformly on x(-) £ U(q). 
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On predicability and causality 

It may appears that Theorem Q] contradicts to the obvious fact that a general process 
cannot be predicted in any sense. For instance, let x\(t) and X2(t) be two processes such 
that x\{t) = X2{t) = for t < and such that xi(t) ^ %2(t) fo r t > 0. Clearly, it is 
not possible to say which process we observe at time t = using the values for t < 0. 
(Some discussion and examples related to the predictability can be found in Chapter 17 
from Higgins (1996)). However, it does not contradict to Theorem [TJ For instance, let 
Xk{t) = (— l) fc ie~* for t > 0; it is easy to verify that the Fourier transforms of the processes 
Xfc(-) do not belong to X(q), q = 1,2. It reflects the lack of causality for these process: the 
values for t > cannot be regarded as continuation of some development started before 
t = 0. In contrast, periodic, almost-periodic, and band-limited processes have causality 
property and therefore can be predicted. 

Theorem [1] says that the predictability can be ensured for some processes other than 
periodic, almost-periodic, or band-limited. In particular, there is some causality for all 
processes covered by this theorem, i.e., some signs of future development are presented in 
the current time. Therefore, Theorem [1] can be interpreted as a new sufficient frequency 
condition of causality for processes that are not periodic, almost-periodic, or band-limited. 
In some cases, this causality makes possible predictability on some fixed finite horizon only. 
In particular, these conditions are more restrictive for longer horizon. 



4 Sufficient conditions of predictability in time domain 



In Theorem [H conditions of predictability are formulated in frequency domain. It can be 
useful to add some sufficient conditions in time domain. 

For C > 0, consider a class M.{C) of processes x(t) G C°°(R) such that there exists 
M > such that 

2 



d k x 

~d¥ { 



<C k M, k = 0,2, 4,6,.... 



L 2 (R) 





d k 1 s M 


2 


d k+1 x t ^ 




( 


dt k ~ l ( ' 


+ 


dt^ 1 ( } 


2 ) 

MR)/ 



<C k M, k = 1,3, 5, 7, 



Vx(-) G M, 



Proposition 1 For any T > and C > 0, M{C) C X{2,T). In particular, for any 
T > and C > 0, the class M{C) is predictable in the weak sense with the prediction 
horizon T. 
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For C > 0, consider a class M{C) of processes x(t) G C°°(R) such that there exists 
M > such that 

2 



L 2 (R) 



<k\C~ k M, k = 0,2,4,6,.... 

<k\C- k M, k = 1,3,5,7, 







2 

+ 
L 2 (R) 


d k+1 x 


2 ) 


( 


dt*- 1 ( } 




£a(R)/ 



Vs(-) G M{c). 



Proposition 2 For any (7 > 2T, AA(C) C X(2,T). In particular, the class N{C) is 
predictable in the weak sense with the prediction horizon T < C/2. 

5 Example: outputs of Gaussian filters 

Let C > and Co > be given. Consider a class of processes Z(C,Q) = {%(■)} such that 
the Fourier transform Z(ioS) = Tz is defined in the class L2 (R) U (R) and |X(iu;)| < C 
for all u such that \uj\ > ui, where z £ Z. 

Let c\ > and v 1 > be given. Consider a set Q = G(ci,V2) of Gaussian filters with 
kernels kg(t) = cexp V) sucn t na ^ ^ > t>i and |c| < ci. 

Let Vg = Vg(C, c, ci, v±) be the set of processes x such that x is a convolution of z 
with a kernel kg, where kg £ r(ci,ui), z G Z(C,uj), i.e., 



x(t) 



kg(t — s)z(s)ds. 



Note that generalized functions v G C(R)* are allowed to be elements of Z (for instance, 
we include delta functions). In that case, Tz is still well defined, and the corresponding 
process x = J r ~ l X(iuj) is well defined in Z<2(R), where X(ioj) = Kg(iu;)Z(iui), Kg(ito) = 
Tkg. 

Proposition 3 (i) Vg C X(l) n X{2); (ii) For q = 1,2, for any T > 0, the class Vg is 
predicted in the weak sense with the prediction horizon T uniformly with respect to the 

norm II • IIl,(r)- 

For example, consider processes 



N 



"<{t) = ^2 c "* ex p 

m=l 



(t - a m f 



(5.1) 



for some constants TV > 0, c m , a m , and v m > 0. By Proposition [3l these processes belong 
to X(l) n X{2), and that any set of these processes such that 



{N < d, \c m \ < C 2 , v rn > C 3 , \a m \ < C 4 } 
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forms a class U{q) with the properties required in Theorem [JJ for any given set of positive 
Ci, C2, C3, C4. Therefore, these processes can be predicted and uniformly predicted in the 
weak sense of Theorem [TJ In particular, it is possible to predict for any T > the values 
ft +T k{t — s)x{s)ds using the values for < t and the predictors defined in the proof of 
Theorem [TJ If a m > and v^ 1 are large enough, then the processes have sharp peaks in 
t > 0, and the values of x(8) are small for < 0, and the impact of the choice of N, c^, and 
afc on y(-)|(_oo.o] is small. However, this impact still exists, ant it makes the prediction at 
time t = possible. 

Special case: extrapolation of the still snapshot of the temperature 

We have regarded x(t) as processes in time with the time variable t. It is the most 
natural model for prediction. However, there are other models where Theorem [TJ can be 
applied. For instance, consider the problem of measurement of the temperature on the 
one-dimensional rod. We consider the still snapshot of the temperature rather than the 
dynamics of the process of heat propagation. Let x(t) be the temperature at the point 
with the coordinate t G R. Let us assume that the temperature is given as (|5.ip with 
c m > 0; this case corresponds to the model when the heat was originated from N point 
sources that were applied at the points t = a m > at past times defined by v m . We assume 
that N,a m ,c m , and v m are unknown and non-observable. Assume that the temperature 
can be measured in the points t S (— 00, 0] only. The problem is to estimate integrals 
Jo k(—s)x(s)ds using the observations at t € [0, +00) only, with T > 0; in fact, it is 
a relaxed version of the extrapolation problem. Theorem [TJ gives the solution, and the 
"predictor" from the proof can be used. 



6 Appendix: Proofs 

The proofs below are very straightforward and do not use the advanced theory of 
spaces; the existence of required predictors is proved by presenting explicit transfer func- 
tions of the predictors with desired properties. 

Let k(-) G /C(T) and K(iu) = J-k. We assume here and below that ueR, 

For 7 e R, 7 > 0, set 

g {p)^T 1 —^p, h(p)±g(p)-Tp, V(p)±e h &. 
1+P 



7 



Lemma 1 (i) V{p) G H°° and K(iio) = V(iuj)K(iuj) can be extended on C + as func- 
tion K{p) £H 2 r\H°°. 

(a) |vx&0I = «p(P3£)- 

(Hi) sup 7>0 \V{iuj)\ < e T K 

(iv) V[iu}) — ► 1 as 7 —* +00 /or a// a; G R. 

fuj For any e > and any £1 > 0, there exists 7 > such that \ V(iui) — 1| < e for all 
LU G [-tt,Q\. 

Proof of LemmaUl Set Q(iu) = e~ iulT K {iuj) , i.e., K(iu) = e iwT K[iuj). Clearly, 

rO rT rT 

Q(iu) = e- iujT e~ iujt k(t)dt = e- iuj{ - T -^k{-6)de = e~ iuJT k{r - T)dr. 

J-T JO JO 

It follows that Q{iuj) can be extended on C + as function Q(p) G H 2 n H°° . 
Further, V{p) = e - Tp e 9 ^ and 

_7-P -7-P + 27 rr, . ^ 2 7P 
<7(j?J = J p = i p = —J p + 1 - 



~f +p 1 + P 1 +V 

It follows that G H°° . Hence #(««;) = V(ia;)iir(iw) = Q(iu)e^ can be extended 
on C + as function K{p) G H 2 n if 00 . Then statement (i) follows. 
Further, 

/ ■ \ „7-*w. (7-iw) 2 . 7 2 -2 7 iu;-w 2 . 

<?uu) = i icj = J — ^ t^iuj = 1 k k ioj. 

7 + no 7^ + or 7 + LO Z 

Then 

Re/ifiu;) = HegUu) = ^ . 

Then statement (ii) follows. 

Let us find the maximum of Reh(p) = Heh(p, 7) in 7 > 0. It suffices to find 7 such 
that Jp-Tte h(iuj) = 0, i.e., such that 



I (g4 ) . w r ») - _ - y . 0. (6.d 

#7 \7 + w / (7 2 + w 2 ) 2 (j 2 +uj 2 ) 2 v ; 



It is easy to see that (|6.ip holds for 7 = L> . For this 7 = |cj|, we have that 



2T7W 2 _ 2T\lo\uj 2 
7 2 + oj 2 ~ 2lo 2 



Re/l M = 72 1 ; ,2 = o, .2 = T M 



Hence (iii) follows. 
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We have that 

2p 



Hp) = Hp, 7) = -Tp- 



i+p 

Hence h(iu,^f) —* as 7 — * +00 for any w £ R. Then statement (iv) follows. 

Further, it follows from continuity of the exponent function that there exists a function 
tp(-) : (0, +00) — > (0, +00) such that if < V"( e ) then \V{iuo) — 1| < e. Let an 

arbitrarily small e > and an arbitrarily large O > be given. Take 7 = 7(e) > 
2Tfi J ty(e)~ 1 , then 

4T 2 t ; 4 

IMw)| 2 = . .„ 2 < v(g) 2 Vw€[-n,f2], 

i.e., 

|V(«w)-l|<e Vwe[-fi,fi], 

Then statement (v) follows. This completes the proof of Lemma [TJ □ 

Proof of Theorem [0 It suffices to present a set of predicting kernels k with desired 

properties. Let V(-) = V(7, •) be as defined in Lemma[TJ Set K(iu) = V(iu>)K (iuS). Let 

the predicting kernels be defined as k(-) = k(-,j(s)) = T~ l K(iuj). 

For x(-) G L g (R), let X(iu) = Fx, Y(iuj) = Ty = K(iu)X(iu). Set Y(iu) = 

K(iu)X(iu) = V{iu)Y(iuj) and y = T^Y . 

Let us prove (i). Since K(iu) £ Loo(R) and X(iu) G L g (R), we have that Y{iuj) = 

K(iu)X{iuj) £ L q (R) and Y G L g (R). By Lemma QJiv), it follows that 

Y(iu>) — > for a.e. w£R as 7 — > +00. (6.2) 

We have that e T ^X G L 9 (R), if (iw) G L DO (R) and 

|#(iw) < \V{iu) - 1| < 2e TH \K(iu)\, lo £ D, (6.3) 

|Y(iw) - Y{iuj)\ < 2e T ^\K(ito)\\X(iuj)\, uj £ D. (6.4) 

By (|6.2p . (|6,4p . and by Lebesque Dominance Theorem, it follows that 

\\ Y - Y h q (R) ^ 0, i.e., \\y - y\\ Lr (R) -> wGR as 7^+00. (6.5) 

Let us prove (ii). Let e > be given, and let f2(e) = e -1 . By Lemma[T{v), there exists 
7 = 7(e) > such that \V(iu) - l\ q < e for all u £ [-J2(e), 0(e)]. For this 7 = 7(e), we 
have 

rM-yM||| ?(R) 



9 



< \V{iu>) -l\ q \K{iui)\ q \X{iuj)\ q duj+ \V(iu) - l\ q \K{iuj)\ q \X{iu)\ q du 

J-Q(e) JD(n(e)) 



< II^MULCR) ( £9 ll^)Hl(R)+ 2 / £ 



e m \X(iuj)\ q dw , (6.6) 

D(fi( £ )) J 



Take £ -> 0. By (|63|) . it follows that ||Y"(iw) - (R) -> and ||y - y|| Lr(R ) -» 

uniformly ovevU(q) n {x(-) : ||X(«u;)|| L? ( R ) < 1}. 

By (|6.5p . (|6.6p . it follows that the predicting kernels fc(-) = /c(-,7(e)) = F K(iu) are 
such as required. This completes the proof of Theorem [TJ □ 

Proof of Proposition [7J Let x(-) G M(C). Since exp(2|cj|T) = Ylk?=o^~liP~ an< ^ 
< + |w| fe+1 )/2, fc > 1, we have that 



/+oo °° {OT\k Z'+oo °° (OTC\^ 

e 2 ^ T \X(iu)\ 2 dw = -=TT / M fc |X(««;)| 2 da> < 27rM E < +°°( 6 - 7 ) 

fc=o fc=0 



It follows that x(-) G ^f(2,T) for all T > 0. By Theorem QJi) , the required predictability 
holds. □ 

Proof of Proposition® Let x(-) G JV(C). Similarly to (|6.7p . we obtain that 



/+oo ;x> (nT\k r+°° x> ('QT'l'j Z~! 

e 2 H T |X(i W )| 2 <L; = £^ / |^| fc |XM| 2 ^<2vrAf^^-^<+oo. 
fc =o fc=o 6 



It follows that x G X(2,T). By Theorem QJi) , the required predictability holds. □ 

Proof of Proposition® It is known that Kg{iuj) = C\/ttv exp(— vtt 2 uj 2 ). It follows that 
e^ T X(ioj) G L 2 (R) n Li(R) and x G #(1) n #(2). It follows also that condition (ii) in 
Theorem Q] is satisfied for class Vg for any T > 0. □ 

Remark 3 T/ie predictors introduced above are stable, since the corresponding transfer 
functions belong to H 2 n H°°. In addition, these predictors are robust with respect to the 
deviations of the process that are small in the weighted norm generated by the definition 
of the space X{q). 

Remark 4 Formally, the predictors described above require the past values of x(s) for all 
s G (— oo,t], but it is not too restrictive, since k(t — s)x(s)ds can be approximated 
by k(t — s)x(s)ds for large enough M > 0. In addition, the corresponding transfer 
functions can be approximated by rational fraction polynomials, and more general kernels 
k can be approximated by kernels from tC{T). 
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